So, you want to scale your computing muscle to train bigger deep learning models. Can your data center handle it? According to Nvidia, which sells more of the specialized chips used in machine learning than any other company, it most likely cannot. These systems often consume so much power, a conventional data center doesn't have the capacity to remove the amount of heat they generate. It's easy to see how customers without infrastructure that can support a piece of Nvidia hardware is a business problem for Nvidia.

If you want to see what the future of iron to support machine learning looks like, then perhaps the best place to look at what the hyperscalers and cloud builders who account for the vast majority of processing and applications in this field are deploying. Or, more precisely, look at the iron that their ODM partners are trying to peddle to other companies that is inspired by what the hyperscalers and cloud builders are using. Inspur, one of the upstart makers of infrastructure that is located in China but which is expanding outwards to North America and Europe, is a good case in point. The company has very good insight into what the Big Four in China – Alibaba, Baidu, Tencent, and either China Mobile or JD.com, depending on how you want to rank numbers four and five – are doing with their vast infrastructure, and it dominates some of these accounts. As we reported back in October 2018, when Inspur was making a push into Open Compute, Inspur has about half of the plain vanilla server shipments and about 80 percent of the GPU accelerated machine learning shipments to the hyperscalers and cloud builders in China. Inspur also works with Microsoft, one of the Big Four in the United States, on its current generation "Project Olympus" servers, the designs of which have been open sourced through the Open Compute Project championed by Facebook alongside some other hyperscale iron that was inspired by Inspur's manufacturing deals with Alibaba and Tencent.

Machine learning is a rising star in the compute constellation, and for good reason. It has the ability to not only make life more convenient – think email spam filtering, shopping recommendations, and the like – but also to save lives by powering the intelligence behind autonomous vehicles, heart attack prediction, etc. While the applications of machine learning are bounded only by imagination, the execution of those applications is bounded by the available compute resources. Machine learning is compute-intensive and it turns out that traditional compute hardware is not well-suited for the task. Many machine learning shops have approached the problem with graphics processing units (GPUs), application-specific integrated circuits (ASICs) – for example, Google TensorFlow – or field-programmable gate arrays (FPGAs) – for example, Microsoft's investment in FPGAs for Azure and Amazon's announcement of FPGA instances.

Let's go through a high-level exploration of the evolution of computational hardware technologies with a focus on applications to machine learning (ML), and using cryptocurrency mining as an analogy. I posit that the machine learning industry is undergoing the same progression of hardware as cryptocurrency did years ago. Machine learning algorithms often consist of matrix (and tensor) operations. These calculations benefit greatly from parallel computing, which leads to model-training performed on graphics cards (rather than only on the CPU). Each step in this progression of technologies produces tremendous performance advantages.